1. Field of the Invention
The present invention relates to an image processing technique for producing a higher resolution image from an image obtained by image capturing.
2. Description of the Related Art
Image pickup apparatuses such as digital still cameras include ones capable of performing optical zooming and digital zooming. For example, Japanese Patent Laid-Open No. 2002-341396 discloses a camera that performs digital zooming between steps of optical step zooming to enable continuous zooming while having such an optical step zooming function.
However, since conventional digital zooming methods perform resolution conversion by image interpolation processing such as linear interpolation processing or bilinear processing, a large digital zoom magnification degrades quality of the image (resolution).
Japanese Patent Laid-Open No. 2007-135133 discloses an image pickup apparatus that produces a high-resolution image by using a plural-image super-resolution technology utilizing plural images captured near a telephoto end of optical zooming, and that performs digital zoom processing on the high-resolution image. Producing such an image whose resolution is higher than that of the captured image by using the plural-image super-resolution technology makes it possible to suppress resolution degradation due to the digital zoom processing.
In addition, methods of the super-resolution technology include an ML (Maximum-Likelihood) method, an MAP (Maximum A Posterior) method, a POCS (Projection Onto Convex Set) method, an IBP (Iterative Back Projection) method and an LR (Lucy-Richardson) method. The LR method disclosed in “Bayesian-based iterative method of image restoration” (W. H. Richardson, J. Opt. Soc. Am. A, 1972, Volume 62, p. 55-59) normalizes an illuminance distribution of an original image or an illuminance distribution of a degraded image to obtain a distribution of a probability density function. The LR method makes it possible to treat a point spread function (PSF) that is a transfer function of an optical system as the distribution of the conditional probability density function. Then, the LR method performs iterative calculation to estimate, on the basis of Bayes statistics, a most probable distribution of the original image for the distribution of the degraded image, by using the distribution of the degraded image and the distribution of the SPF (that is, the distribution of the probability density function).
International Publication WO2006-041127 discloses a method for restoring a more accurate original image by using an optical transfer function (OTF) in place of the PSF used in the LR method, the OTF easily containing an accurate phase characteristic.
However, the plural-image super-resolution technology disclosed in Japanese Patent Laid-Open No. 2007-135133 requires storing of a lot of images (for example, ten images) to a memory, which increases a required capacity of the memory.
Moreover, the conventional plural-image super-resolution technology requires knowing of an accurate transfer characteristic (transfer function) of an image taking optical system in order to estimate an entire image with high resolution from only a low-resolution image. However, it is generally difficult to accurately evaluate the transfer characteristic of the image taking optical system since the transfer characteristic depends on parameters such as various aberrations of lenses constituting the image taking optical system, an object illumination wavelength and an aperture size of an image-pickup element. Moreover, it is not general to individually measure and use the transfer characteristic of the image taking optical systems provided to each image pickup apparatus. Even if estimating a high-resolution image based on an inaccurate transfer characteristic of the image taking optical system, it is impossible to produce a good high-resolution image.